( 

United States Patent m 



Moed et al. 



mi 



US005770841A 
[ii] Patent Number: 5,770,841 
[45] Date of Patent: Jun. 23, 1998 



[54] SYSTEM AND METHOD FOR READING 
PACKAGE INFORMATION 

[75] Inventors: Michael C. Moed, Roswell, Ga.; 

Johannes A. S. Bjorner, Woodbury, 
Conn. 

[73] Assignee: United Parcel Service of America, 
Inc., Atlanta, Ga. 

[21] Appl. No.: 536,865 

[22] Filed: Sep. 29, 1995 

[51] Int. CI. 6 G06K 7/10 

[52] VS. CI 235/375; 235/454 

[58] Field of Search 235/375, 454 

[56] References Cited 

U.S. PATENT DOCUMENTS 

3,949,363 4/1976 Holm. 

4,403,339 9/1983 Wevelsiep et al 382/44 

4,411,016 10/1983 Wakcland 382/62 

4,516,265 5/1985 Kizu et al 382/48 

4,776,464 10/1988 Miller et al 209/3.3 

4,832,204 5/1989 Handy et al 209/3.3 

4,921,107 5/1990 Hofer 209/546 

5,031,223 7/1991 Rosenbaum et al 382/1 

5,120,940 6/1992 Willsie 235/462 

5,124,692 6/1992 Sasson 340/727 

5,189,292 2/1993 Batterman et al 235/494 

5,307,423 4/1994 Gupta et al 382/11 

5,308,960 5/1994 Smith et al 235/454 

5,311,999 5/1994 Malow et al 209/583 



5,327,171 7/1994 Smith et al 348/223 

5^87,783 2/1995 Mihm et al 235/375 

5,420,403 5/1995 Alium et al 235/375 

5,478,990 12/1995 Montanari 235/375 

FOREIGN PATENT DOCUMENTS 

0 647 479 A2 12/1994 European PaL Off. . 
0 647 479 A3 12/1994 European PaL Off. . 
39 42 932 Al 6/1991 Germany . 

Primary Examiner — Harold Pitts 

Attorney, Agent, or Firm — Jones & Askew, LLP 

[57] ABSTRACT 

I A system for reading package inform ation includes an 
imaging system and a label aecoarag system The imaging 
system captures an image of a package surface that includes 
a. machine readable code such as a bar code and an alpha- 

£ numeric destination address. The label decoding system 
locates and decodes the machine readable code and uses 
OCR techniques to read the destination address^ The desti- 
nation address is validated by comparing the decoded 
address to a database of valid addresses. If the decoded 

|o address is invalid, an image of the destination address is 
displayed on a workstation and an operator enters the correct 
address. The system forms a unified package record by 
combining the decoded bar code data and the correct des- 
tination address data. The unified package record is used for ^ 

tr subsequently sorting and tracking the package and is stored 
in a database and applied to a label that is affixed to the 
package. 
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SYSTEM AND METHOD FOR READING 
PACKAGE INFORMATION 

TECHNICAL FIELD 

The present invention relates to package tracking systems, 
and more particularly relates to systems for automatically 
reading and decoding package information such as machine 
readable codes and alphanumeric destination information. 

BACKGROUND OF THE INVENTION 

Small package delivery companies such as the assignee of 
the present invention may handle as many as several million 
packages each day. In order to improve the efficiency and 
accuracy with which this volume of packages is handled, 
these companies increasingly rely on automated package 
sorting and routing facilities. Small package delivery com- 
panies also desire to obtain package related information in 
order to better manage their operations and to provide a 
variety of shipping related information to their customers. 

The process of sorting and tracking packages as they 
proceed through a package transportation system requires 
that each package bear two types of information. First, each 
package must provide a destination address. Second, each 
package must include a tracking number that uniquely 
identifies it from other packages in the system. 

The destination address is required in order for the 
package delivery company to know where the package is 
going. The destination address, which includes alphanu- 
meric text, is typically written on the package or printed on 
a label that is affixed to the package. For addresses in the 
United States, the destination address includes a street 
address, city, state and zip code. 

The tracking number, which consists of a series of alpha- 
numeric characters, uniquely identifies each package in the 
package transportation system. In most cases, the tracking 
number is affixed to the package in the form of a machine 
readable code or symbol such as a bar code. The machine 
readable code is read by electronic code readers at various 
points in the transportation system. This allows the package 
delivery company to monitor the movement of each package 
through its system and to provide customers with informa- 
tion pertaining to the status and location of each package. 

The importance of collecting package related data has led 
to the development of a variety of devices for reading bar 
codes and other machine readable codes. These devices 
include hand held readers used by employees when they 
pick up or deliver packages, and over-the-belt cameras that 
are mounted over conveyor belts in order to read machine 
readable codes as the packages move through the delivery 
company's terminal facilities. 

In some cases, shippers may also print and affix labels 
including two-dimensional machine readable codes that 
include both package identification information and desti- 
nation address information. These dense codes are read by 
over-the-belt cameras and the information is used to track 
and sort the package. However, for packages that enter the 
delivery company's system without such labels, there is no 
efficient, automatic way to prepare such labels and affix 
them to packages. 

Optical character recognition (OCR) technology has also 
improved to the point where it is feasible to automatically 
read and decode printed destination address data. The 
assignee of the present invention has developed over-the- 
belt camera systems that can be used to capture and decode 
bar codes and text as packages travel beneath the camera on 
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a conveyor belt. The ability to read and decode destination 
address data is useful because it facilitates automatic sorting 
and routing of packages in the delivery system. 
Although OCR systems are becoming more common, 

5 there are often difficulties associated with decoding data 
from packages moving on a conveyor belt at a high rate of 
speed. Current bar code decoding techniques provide for 
using a variety of algorithms for scanning an image and 
locating and decoding a bar code. These techniques are very 

10 accurate, in part because of the use of checksums and other 
techniques to ensure the reliability of the bar code decoding 
process. OCR techniques typically apply a variety of decode 
algorithms to a string of text in order to accurately decode 
the text. However, there remains the possibility that the 

15 address data may be improperly decoded. Furthermore, it is 
difficult to detect an improperly decoded address because 
OCR decoding does not employ checksums and other tech- 
niques that are available to verify the accuracy of machine 
readable codes. 

20 Therefore, there is a need in the art for a system that reads 
and decodes bar codes and text, and which verifies the 
accuracy of the destination address data. Furthermore, there 
is a need for a system that provides a method for correcting 
improperly decoded destination address data, and for com- 

25 bining the destination address data and the decoded bar code 
data to form a unified package record, which may be used to 
track and sort the package as it moves through the package 
delivery system. 

30 SUMMARY OF THE INVENTION 

The present invention satisfies the above-described need 
by providing a system and method for reading package 
information. In the system of the present invention, a 

35 package bears at least one label that includes information 
indicia such as a destination address and a machine readable 
symbol (for example, a bar code or two-dimensional dense 
code) bearing a package identification number. As packages 
move along a conveyor belt, an image of each package is 
captured and the indicia are decoded. The decoded destina- 
tion address is validated by checking a database of valid 
addresses. If the decoded address is invalid, an image of the 
address is displayed on an image display workstation, and an 
operator enters the correct destination address. The symbol 

45 data and destination address are combined to form a unified 
package record, which may be used to sort and track the 
package. The unified package record may be stored in a 
database or printed on a label and affixed to the package. 
Generally described, the present invention provides a 

50 method for reading package information from a package that 
includes first and second information indicia. The method 
includes capturing an image of the package. The captured 
image includes the first information indicia and the second 
information indicia. The first information indicia is located 

55 and decoded to provide first package data. The second 
information indicia is located and decoded to provide second 
package data. The first and second package data are then 
combined to form a unified package record. The unified 
package record may be stored in a database or printed on a 

60 label and affixed to the package. 

In another aspect, the present invention provides a method 
for reading and verifying package information from a pack- 
age. The method includes capturing an image of the 
package, which includes information indicia. The informa- 

65 tion indicia is located and decoded to provide first package 
data. The first package data is verified to determine whether 
it is valid. If not, the image of the information indicia is 
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displayed on a workstation. Manually entered first package image display workstation, along with a list of possible 

data is then received from an operator at the workstation. addresses from the database. An operator reads the destina- 

In yet another aspect, the present invention provides a tion address data from the display and manually enters it into 

system for reading package information from a package, the computer terminal or selects the correct address from a 

which includes first and second information indicia. The * displayed list of possible addresses. After the destination 

system includes an imaging system with a camera for address has been validated or manually entered, the bar code 

capturing an image of the package, and a label decoding data and destination address data are combined to form a 

system for processing the image. A printer is provided for unified package record, which provides efficient means for 

printing a label to be affixed to the package. Hie label automatically tracking and sorting packages This data may 

decoding system is programmed to locate and decode the ™ be stored in a database or printed on labels and affixed to the 

first information indicia in the image, thereby providing first package. 

package data. The label decoding system also locates and Before describing the present invention in additional 

decodes the second information indicia in order to provide detail, it is useful to discuss the nomenclature of the speci- 

second package data. The first and second package data are fication. Portions of the detailed description that follows are 

combined to form a unified package record, which may be 15 represented largely in terms of processes and symbolic 

printed by the label printer. representations, of operations performed by computer 

More particularly described, the label decoding system of components, including a central processing unit (CPU), 

the present invention includes an image display workstation. memory storage devices for the CPU, and connected display 

The system is operative to determine whether the second devices. These operations include the manipulation of data 

package data is valid and, if not, display the image on a 20 by the CPU and the maintenance of these data within data 

workstation. The system receives manually entered second structures resident in one or more of the memory storage 

package data from the workstation, and forms the unified devices. The symbolic representations are the means used by 

package record from the first package data and the manually those skilled in the art of computer programming and 

entered second package data. computer construction to most effectively convey teachings 

, . . - A Cl . . . 25 and discoveries to others skilled in the art. 

It is therefore an object of the present invention to provide 

a system that reads and decodes all relevant package data For ^ purposes of this discussion, a process or portions 

from a package thereof may be generally conceived to be a sequence of 

, . . . A , computer-executed steps leading to a desired result. These 

It is another object of the present invention to verify the P require physical manipulations of physical 

accuracy of the decoded package data. 30 qua ^ ies ^Llly, t^^t necessarily, these quantities 

It is another object of the present invention to facilitate the take ^ form of G i cctr i ca ^ magnetic, or optical signals 

correction of incorrectly decoded package data. capable of being stored, transferred, combined, compared, or 

It is another object of the present invention to provide a otherwise manipulated. It is conventional for those skilled in 

unified package record including relevant package data. the art to refer to these signals as bits, values, elements, 

35 symbols, characters, terms, objects, numbers, records, files 

BRIEF DESCRIPTION OF THE DRAWINGS or me ^ It should be kept m mmd> however, that these and 

FIG. 1 is a block diagram of a system for reading package similar terms should be associated with appropriate physical 

information in accordance with the present invention. quantities for computer operations and that these terms are 

A . r , a A . . merely conventional labels applied to physical quantities 

FIG. 2 is a diagram of a parcel including a fluorescent ink 40 ^ exist md durin operation of me com puter. 

fiduciary mark located within the destination address block . 

of the arcel should also be understood that manipulations within the 

" ,. , computer are often referred to in terms such as adding, 

FIG. 3 is a flow diagram of the process for reading arf movin etc which m oflen associated with 

package information earned out by the system of FIG. 1. manual operations performed by a human operator. In most 

FIG. 4 is a flow diagram of the preferred method for cases> j t ^1 ^ apparent that these steps are performed by 

processing image data provided by the imaging system that a com puter without requiring input from an operator. In 

forms a part of the system,of FIG. 1. some cases, the operations described herein are machine 

FIG. 5 is a flow diagram of the preferred method for operations performed in conjunction with a human operator 

correcting incorrectly decoded destination address data. 5Q that interacts with the computer. The machines used for 

performing the operation of the present invention include 

DETAILED DESCRIPTION OF THE general purpose digital computers or other similar comput- 

PREFERRED EMBODIMENT mg devices. 

The present invention provides a novel system and 1° addition, it should be understood that no particular 

method for reading package information. Generally 55 programming language is provided, and that the programs, 

described, the system includes an imaging system that processes, methods, etc. described herein are not limited to 

provides a digital image of a surface of a package that is any particular computer or apparatus Those skilled in the art 

moving on a conveyor belt. The image includes a bar code will appreciate that there are many computers and operating 

and destination address that are provided on the package systems which may be used in practicing the instant inven- 

surface. A label decoding system processes the image from 60 tion and therefore no detailed computer program could be 

the imaging system and decodes the bar code and the . provided which would be applicable to these many different 

destination address data. The destination address data is systems. Each user of a particular computer or operating 

validated by checking the address against the United States system will be aware of the program modules and tools that 

Postal Service's ZIP+4 database, which contains all of the are most appropriate for that user's needs and purposes, 

valid addresses in the United States. If the destination 65 Referring now the drawings, in which like numerals 

address was decoded incorrectly, the portion of the image represent like elements throughout the several figures, the 

that includes the destination address is displayed on an present invention will be described. 
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THE SYSTEM FOR READING PACKAGE camera 16 to accurately focus on the top surface of the 

INFORMATION package 20c as it moves beneath the camera. The illumina- 
tion source 28 illuminates the top surface of the package 20c 

FIG. 1 illustrates a system 10 for reading and decoding as it passes beneath the high resolution camera 16. The 

package information as packages trawl on a conveyor belt. 5 location and orientation information are provided to the 

The system 10 includes an imaging system 12 and a label label decoding system 14 along with the image from the high 

decoding system 14. Generally described, the preferred resolution camera 16. 

imaging system 12 is a two-camera system that includes a xh e conveyor belt system is used to transport packages 
high resolution over-the-belt (OTB) camera 16 and a fidu- through a terminal facility. In the preferred system 10, the 
ciary mark detector 24, which includes the second camera. 1Q conveyor belt 18 is 16 inches wide and carries up to 3,600 
FThe high resolution OTB camera 16 and fiduciary mark packages per hour while moving at a rate of up to 100 feet 
1 detector 24 are mounted above a conveyor belt 18 that pcr mmu te. The packages 20a-c vary in height and may be 
[carries packages 20a-c in the direction of arrow 22. arbitrarily oriented on the conveyor belt 18. The conveyor 
Together, the high resolution OTB camera 16 and fiduciary belt ig m0V es each package beneath the fiduciary mark 
mark detector 24 ascertain the position and orientation of a 15 detector 24 and high resolution camera 16 in single file, and 
fluorescent ink fiduciary mark located within a destination with some amount of space between them. The packages are 
address block on the surface of a package, capture an image separated by a device known as a singulator. A suitable 
of the top surface of the package, and provide the image and singulator is described in U.S. Pat. No. 5,372,238 to Bonnet, 
the location and orientation of the fiduciary mark to the label entitled "Method and Apparatus for Singularizing Objects." 
decoding system 14. The label decoding system 14 includes 2Q belt 18 includes a belt encoder 44 that is 
general purpose and high performance computers and data used to determine the speed and position the associated 
storage facilities. The label decoding system 14 is connected conveyor belt. Those skilled in the art will appreciate that the 
to an image server 29, which is connected to at least one speed and pos i t i on 0 f the conveyor are needed in order to 
image display workstation 30a-c, and to a label printer 32. synchronize the position of the fiduciary mark, the package 
The label decoding system 14 locates and decodes machine 25 hcight information, and the position of the package as it 
readable package identification data (e.g., a bar code) and passes beneath the high resolution camera 16. The belt 
destination address data contained in the image. This pack- encoder supplies a signal indicating the speed of the con- 
age identification data and destination address data are veyor 18 t0 the fiduciary mark detector 24 and the high 
combined to form a unified package record, which may be resolution camera 16. The signal from the encoder is used to 
stored in a database or printed in machine readable form on 3Q produce a line clock signal that is used to trigger cycles of 
a label and affixed to the package. the fiduciary ma rk detector's low resolution camera (i.e., 
FIG. 2 illustrates the top surface 34 of a package 20 that exposures of the line of CCD pixels comprising the low 
is processed by the preferred system 10. The top surface 34 resolution camera). Each cycle captures a row of the image 
of each package 20 includes package tracking information in of the surface of a parcel as it moves past the fiduciary mark 
the form of a machine readable code or symbol such as a bar 35 detector 24. The belt encoder 44 is selected to provide a 
code 36. The package tracking information represented by pulse for each cycle of the high resolution camera 16. Those 
the bar code uniquely identifies the package and distin- skilled in the art will appreciate that the signal from the 
guishes it from the other packages in the delivery system. encoder allows the line images captured by the fiduciary 
The top surface of the package also includes' a destination mark detector 24 and high resolution camera 16 to be 
address 38, which typically consists of alphanumeric text 40 assembled by the label decoding system 14 into two- 
arranged in two or more lines. The destination address 38 is dimensional images with the correct aspect ratios. A more 
located in an area referred to as the destination address block detailed description of the interaction between an OTB 
40. A fiduciary mark such as fluorescent ink fiduciary mark camera, conveyor belt, height information processor, and 
42 is located approximately in the center of the destination belt encoder is provided in U.S. Pat. No. 5,291,564 to Shah, 
address block 40 in the same area as the text defining the 45 entitled "System and Method for Acquiring an Optical 
destination address. The fiduciary mark 42 is applied to the Target," which is incorporated herein by reference, 
destination address block 40 by the shipper or by an agent A suitable fiduciary mark detector is described in pending 
of the small package delivery company. This may be accom- U.S. application Ser. No. 08/419,176, filed Apr. 10, 1995, 
plished by using a rubber stamp in the shape of the desired and entitled "Method for Locating the Position and Orien- 
fiduciary mark to apply fluorescent ink to the package 50 tat j oa D f a Fiduciary Mark," which is assigned to the 
surface. Those skilled in the art will appreciate that other assignee of the present invention and is incorporated herein 
types of fiduciary marks may be used. by reference. The fiduciary mark detector 24 includes a low 
Referring again to FIG. 1, the components and operation resolution CCD camera, a video processor, and an ultraviolet 
of the imaging system 12 and the label decoding system 14 light source for illuminating the fluorescent ink that forms 
will be described in additional detail. In addition to the high 55 the fiduciary mark. The conveyor belt 18 moves a package 
resolution OTB camera 16 and fiduciary mark detector 24, 20a through the field of view of the low resolution CCD 
the imaging system 12 includes a package height sensor 26, camera. The video processor controls the operation of the 
and an illumination source 28. As packages are transported low resolution camera and sequentially transmits a one-bit 
by the conveyor belt 18 the packages 20a-c first pass under (i.e., black/white) video signal corresponding to the image 
the fiduciary mark detector 24, which detects a fiduciary 60 captured by the low resolution camera to the label decoding 
mark in order to determine the location and orientation of the system 14. The preferred low resolution camera is a low 
destination address block. The package height sensor 26 is resolution, monochrome, 256 pixel line-scan type camera 
a commercially available light curtain, and is used to deter- such as a Thompson TH7806A or TH7931D. The ultraviolet 
mined the height of the package before it passes beneath the light source illuminates the package 20a as it is conveyed 
high resolution OTB camera 16. The height information 65 through the viewing area of the low resolution camera, 
from the height sensor 26 is used by the high resolution which captures an image of the surface of the package 20a. 
camera's focusing system. This permits the high resolution The low resolution camera is fitted with a commercially 
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available optical filter that transmits yellow/green light such 
as that emitted by fluorescent ink exposed to ultraviolet light 
and attenuates light in other portions of the visible spectrum. 
The low resolution camera is thus configured to be respon- 
sive to the ycllow/grccn light emitted by the illuminated 5 
fiduciary mark, and not to the other indicia found on the 
package surface. More specifically, the optical filter causes 
the low resolution camera to be responsive to the yellow/ 
green light emitted from the commercially available 
National Ink No. 35-48-J (Fluorescent Yellow) in response 
to ultraviolet light. 

Referring again to FIG. 2, the preferred fiduciary mark 42 
will be described in additional detail. The preferred fiduciary 
mark 42 comprises two fluorescent non-overlapping circles 
of different diameter. As used herein, a circle means either 15 
an annulus or the area bounded by an annulus. The fiduciary 
mark 42 includes a large circle and a small circle oriented 
such that a vector from the center of large circle to the center 
of the small circle is oriented approximately in the same 
direction as underlying text of the destination address 38. ^ 
The position of the fiduciary mark 42 is defined to be the 
mid-point of the vector. It will be clear to those skilled in the 
art that alternative embodiments might include locating the 
fiduciary mark elsewhere on the parcel in a known relation 
to a text bearing area, or in a different known relationship to ^ 
the underlying text. The fiduciary mark 42 is typically 
applied to a parcel using a conventional rubber stamp and 
fluorescent ink after the destination address 38 has been 
affixed to the parcel. It will be appreciated that the fiduciary 
mark 42 might be carried on a label, preprinted upon the 30 
parcel, or might be carried upon a transparent envelope into 
which an address label is placed. 

For the preferred fiduciary mark 42, the diameter of the 
large circle is approximately 3 A of an inch, the diameter of 
the small circle is approximately Vis of an inch, and the 35 
distance separating them is approximately Va of an inch. It is 
noted that a limit is imposed upon the size of the fiduciary 
mark 42 by the resolution of the low resolution camera that 
forms a part of the fiduciary mark detector 24. For example, 
the fiduciary mark 42 may be made smaller if the low 40 
resolution camera has a higher resolution, and the resolution 
of camera may be reduced if the fiduciary mark is made 
larger. 

Those skilled in the art will appreciate that a fiduciary 
mark can be any mark that identifies the location of the 45 
destination address and that the preferred fiduciary mark 
comprising two circles is simply one of a variety of possible 
choices. Those skilled in the art will also appreciate that 
although the preferred fiduciary mark indicates the location 
and orientation of the destination address it is possible to use so 
a fiduciary mark that indicates only location. In such a case, 
the orientation would be determined by applying an appro- 
priate processing technique to the image of the destination 
address-block. 

The preferred system 10 also defines a region of interest 55 
defined with respect to the fiduciary mark 42. The region of 
interest is defined in terms of the high resolution camera to 
be a 1 k by 1 k square (i.e., 1,024 pixels by 1,024 pixels, 
which is equivalent to approximately four inches by four 
inches) centered on the defined position of the fiduciary 60 
mark 42. The label decoding system 14 determines the 
position and orientation of the fiduciary mark 42 and defines 
the region of interest with respect to the position of the 
fiduciary mark 42. The label decoding system then creates 
and stores a high resolution text image within the region of 65 
interest from the data captured by the high resolution camera 
16. In this manner, only a relatively small portion of the data 
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captured by the high resolution camera 16 is processed in 
order to decode the destination address data. 

The package height sensor 26 is a commercially available 
light curtain, and is used to determined the height of the 
package before it passes beneath the high resolution OTB 
camera 16. The height information from the height sensor 26 
is used by the high resolution camera's focusing system. 

The preferred illumination source 28 includes an u asym- 
metrical elliptical reflector. The reflector is shaped by first 
and second elliptical surfaces. The first and second elliptical 
surfaces share a common first focus, along which the light 
source is located. The first and second elliptical surfaces 
have different second foci. Thus, half of the elliptical surface 
concentrates the fight at one level and the other half con- 
centrates the light at a second level. Together, the first and 
second elliptical surfaces develop intense illumination 
between their respective second focal axes. 

The high resolution camera 16 is preferably a 
monochrome, 4,096 pixel line-scan type camera such as one 
using a Kodak KLI-5001 CCD chip. Each pixel measures 
approximately 7 micronsx7 microns. The CCD array is 
sufficiently wide to scan the entire width of the conveyor 
belt. The image of the package is captured one "slice** at a 
time as the package moves beneath the camera. The high 
resolution camera 16 transmits an eight-bit gray-scale video 
signal corresponding to the captured image to the label 
decoding system 14. Illumination source 28 provides bright 
white tight in order to illuminate the package as it is 
conveyed through the viewing area of the high resolution 
camera 16, which captures an image of the surface of a 
package. The high resolution camera 16 is responsive to a 
grayscale light pattern such as that reflected by black ink text 
on the surface of the package 20c. The high resolution 
camera 16 is relatively unresponsive to light such as that 
reflected by fluorescent ink when illuminated by white light. 
More specifically, the commercially available National Ink 
No. 35-48-J (Fluorescent Yellow) is substantially invisible 
to the high resolution camera 16 when illuminated- by the 
white light source 28. 

Suitable high resolution camera systems are described in 
U.S. Pat. Nos. 5,327,171 to Smith et al., entitled "Camera 
System Optics" ("the '171 patent"), and 5,308,960 to Smith 
et al., entitled "Combined Camera System," and in allowed 
U.S. application Ser. No. 08/292,400, filed Aug. 18, 1994, 
entitled "Optical Path Equalizer" ("the Optical Path Equal- 
izer application"), all of which are assigned to the assignee 
of the present invention and incorporated herein by refer- 
ence. 

The '171 patent describes an OTB camera system for 
capturing images of packages as they move beneath the 
camera on a conveyor belt. The system described in the '171 
patent includes an illumination source, a belt encoder for 
determining the speed and position of the conveyor belt, and 
a processing subsystem that searches for a number of 
different acquisition targets. 

The Optical Path Equalizer application describes an OTB 
camera with an optical system that equalizes the path 
between the OTB camera and the package located beneath 
the camera. This allows the camera to accurately focus on 
the package surface regardless of the package's height, and 
also maintains an approximately constant image size regard- 
less of the height of the package. The optics assembly 
includes a pair of movable mirrors and an array of fixed 
mirrors. The movable mirror are mounted on pivot pins and 
are rotated by one or more actuators. The array of fixed 
mirrors includes a plurality of mirrors positioned at increas- 
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ing distances from the movable mirrors as to provide a sort. In the preferred system 10, the unified package record 
plurality of different optical path lengths between the camera is printed in machine readable dense code, such as the codes 
and the package surface. The Optical Path Equalizer appli- described in U.S. Pat. No. 4,896,029 to Chandler et al., 
cation also describes the use of a height sensing device such entitled "Polygonal Information Encoding Article, Process 
as a commercially available light curtain. The data from the 5 and System" and U.S. Pat. No. 4,874,936 to Chandler et al., 
height sensing device is used to determine the optical path entitled "Hexagonal, Information Encoding Article, Process 
length of the variable optical subsystem, and System." Those skilled in the art will appreciate that the 
The label decoding system 14 processes the data provided number of label printers will depend on the configuration of 
by the imaging system 12. The label decoding system 14 the conveyor system, the number of packages moving 
includes input/output devices for receiving data from the through the system, and other factors, 
fiduciary mark detector 24 and the high resolution camera THE PREFERRED METHOD FOR READING 
16. The label decoding system includes both general purpose PACKAGE INFORMATION 
computers and high performance computers. The high per- 
formance computers, such as Adaptive Solutions CNAPS ^ Preferred method for reading package information 
processor and Imaging Technologies 150/40 processor, are 15 wil1 now be dossed in conjunction with FIGS. 3-5. As 
used to run that OCR algorithms that are used to decode the described above, the system 10 is operative for capturing an 
alphanumeric destination address data. The general purpose ima g e of a Package as it travels on a conveyor belt, and 
computers, such as Heurikon Nitro 60 and Heurikon detecting and decoding a bar code and OCR address data 
HKV4D computers, are used to process the location and a PP ear on the P^ge. The OCR data is validated and, 
orientation data from the fiduciary mark detector 24 and to ^ tf not accurate, is displayed on a terminal where an operator 
decode detect and decode the bar code that includes the -can- manually enter the address data. The decoded bar code 
package tracking information. The label decoding system data ^ address data are combined to form a unified 
includes storage devices such as memory, disk drives and package record, which is subsequently used to sort and track 
tape drives. The label decoding system may also be con- ^ G package. 

nected to other computing equipment that is used for pack- 25 FIG * 3 ^ a flow ^gram illustrating the preferred method 

age tracking, billing, etc. 300 for reading package information. The steps that form the 

The label'decoding system 14 is connected to a image method 300 are carried out b y the various equipment that 

server 29, which is connected to a network that includes a forins a P art of me svstcm 10 for reading package informa % 

plurality of image display workstations 30a-c. If the label * on - ^ method 300 begins at step 302 by determining the 

decoding system is unable to verify a decoded destination 30 locatlon and orientation of the destination address block. In 

address by reference to the U.S. Postal Service's ZIP+4 me preferred system, this is accomplished as the package 

database, the system 10 displays the destination address moves ^neath the fiduciary mark detector 24, which is 

image on one of the image display workstations 30a-c, described above in conjunction with FIGS. 1 and 2. The 

where it is viewed by an operator. The displayed destination coordinate and orientation information from the fiduciary 

address image is accompanied by the closest addresses from 35 maik detector are provided to the label decoding system 14, 

the database. The operator than reads the address on the where tne y are used to process the image that is provided by 

display and manually enters the correct address or selects the me resolution camera 16. 

correct address from the list of the closest addresses. Thus, After the package is scanned by . the fiduciary mark 

the image display workstation must include a display, a detector, the package height is determined by the package 

processor, input means such as a keyboard, and input/output 40 hei S ht sense* 26 at step 304. At step 306 a high resolution 

means for communication data to and from the label decod- imzge of the top of the package is captured by the high 

ing system. The preferred image display workstations 30a-c resolution OTB camera 16 as the package passes beneath the 

are IBM compatible personal computers based on Intel high resolution camera. This image is provided to the label 

Corporation's PENTIUM processor and running Microsoft decoding system 14. The high resolution camera 16 uses the 

Corporation's WINDOWS NT operating system. Those 45 package height data from the package height sensor 26 to 

skilled in the art will appreciate that the image display adjust the focal length of the camera and ensure that the 

workstations may include any computer imaging system or camera is properly focused regardless of the height of the 

other computer image processor capable of receiving and package. 

processing pixel images and other information at high rates At step 308 the label decoding system 14 processes the 

of speed, and that the number of such image display work- 50 data fr° m the belt encoder 44, the fiduciary mark detector 

stations used in a facility will depend on the volume of 24, and the high resolution camera 16. Generally described, 

packages moving through the system and various other the processing performed by the label decoding system 

factors. Those skilled in the art will also appreciate that the includes locating and decoding the bar code, locating and 

.image server 29 may be any computer or network server . decoding the destination address, verifying the accuracy of 

capable of being connected to the image display worksta- 55 the destination address, and receiving a manually entered 

tions and capable of transferring and processing pixel destination address if needed. The particular steps involved 

images at high rates of speed. in processing the data are described below in conjunction 

The label decoding system is also connected to at least FIG. 4. 

one label printer 32. As mentioned briefly above, the At step 310 the bar code and destination address data are 

decoded package identification information and destination 60 combined to form a unified package record, which is stored 

address are combined to form a unified package record, in a database or printed on a label and affixed to the package 

which may be used to facilitate the track and sorting of the at step 312. The data contained in the unified package record 

package throughout the delivery system. While the unified is subsequently used for sorting and tracking the package as 

package record may be stored in a database, it may also be it moves through the delivery company's system. The 

printed on a label and automatically affixed to the package 65 method 300 terminates at step 314. 

as it travels on the conveyor belt. The preferred label printer FIG. 4 is a flow diagram illustrating the preferred method 

32 is an automatic label applicator, manufactured by Accu- 308 for processing image data. This method is carried out by 
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the label decoding system 14 and forms a part of the method and scope of the present invention. Furthermore, those 

300 of FIG. 3. The method 308 begins at step 400 when the skilled in the art will appreciate that the fiduciary mark 

label decoding system receives the data from the belt detector described above may be replace with other appa- 

encoder 44, the fiduciary mark detector 24 and the high ratus for indicating and detecting the location and orienta- 

resolution OTB camera 16. As described above, the high 5 tion of an indicia on a package, such as the systems 

resolution camera provides an image of the top of a package. described in U.S. Pat. Nos. 4,516,265 to Kizu et al. and 

The image includes a bar code 36 and a destination address 5 103 439 to Miette. 

38. The fiduciary mark detector provides data indicating the ' A ' An£ matUn A ^ wo , 

. A . . . . , . r.i. j *■ ** j j ui t An At step 406 the method performs adaptive thresholding on 

location and orientation of the destination address block 40. . t , ■ , iU 5™ 

JM « A * , the ROI image. This technique involves binanzing the ROl 

Al step 402 the label decoding system 14 locates and 10 ^ and ^ djfferent binarized im ^ 

decodes the bar code 36 or other machine readable symbol ^ different mreshold vaIues . ^ threshold values 

which is contained m the image provided by the high are detennined b measuring the contrast and relative 

resolutton camera 16. Those skilled m the art will be familiar brightness of me RO i image. 

with various systems and methods for locating and decoding . , . .. ... 

bar codes. Suitable methods for locating and decoding the 1S A \ &{ \ m * e j^iec images resulting fom step 406 are 

bar code 36 are described in U.S. Pat. No. 5343,028 to At step 410 the best of the three run 

FigareUa et al., entitled "Method and Apparatus for Detect- len S th encoded ma 8 es 15 8elccted for 

ing and Decoding Bar Code Symbols Using Two- Suitable methods for carrying out steps 406, 408, 410 are 

Dimensional Digital Pixel Images," U.S. Pat. No. 5^52,878 described in commonly owned U.S. application Sen No. 

to Smith et al., entitled "Method and Apparatus for Decod- M 08/380,732, filed Jan. 31, 1995, entitled "Method and Appa- 

ing Bar Code Symbols Using Independent Bar and Space ratus for Separating Foreground From Background in 

Analysis," U.S. Pat. No. 5,412,196 to Surka, entitled Containing Text," which is incorporated herein by 

"Method and Apparatus for Decoding Bar Code Images reference. 

Using Multi-Order Feature Vectors," and U.S. Pat. No. At step 412 the label decoding system performs a coarse 

5,412,197 to Smith, entitled "Method and Apparatus for 2S rotation of the selected run length encoded image. The 

Decoding Bar Code Symbols Using Gradient Signals," all of coarse rotation is. the first of a two-step process that is 

which are assigned to the assignee of the present invention designed to make the ROI image appear horizontal in order 

and incorporated herein by reference. Those skilled in the art to simplify the separation of the characters. Generally 

will appreciate that the machine readable code or symbol described, the information derived from the fiduciary mark 

decoded by the label decoding system may include a bar 30 indicates the orientation of the destination address block and 

code or a two-dimensional code. how far off of horizontal it is. The coarse rotation is the first 

At step 404 the method 308 begins the process of locating step toward rotating the image to where the destination 

and decoding the destination address. Steps 404 through 422 address appears horizontal. 

are associated with the application of optical character The preferred method for rotating the ROI image is 

recognition (OCR) techniques to the image provided by the 35 described in commonly owned U.S. application Sen No. 

high resolution camera 16. This process is carried out in 08/507,793, filed Jul. 25, 1995, entitled "Method and Sys- 

parallel with decoding the bar code (step 402). tem for Fast Rotation of Run-Length Encoded Images," 

At step 404 the label decoding system selects a subimage which is incorporated herein by reference. Those skilled in 

of the package surface from the image provide by the high ^ art will appreciate that he coarse rotation process is 

resolution camera 16. In the preferred system, this subimage 40 relatively quick and rotates the image to within ±7 degrees 

is referred to as a region of interest (ROI), which is defined °f horizontal. y 

with respect to the fiduciary mark 42. In terms of the image At step 414 the label decoding system identifies the lines 

from the high resolution camera, the region of interest is a of text that are contained in the destination address block 40. 

1 k by 1 k square (i.e., 1,024 pixels by 1,024 pixels, which his is accomplished by subsampling the image by a factor of 

is equivalent to approximately four inches by four inches) 45 3 in the x and y directions, executing a connected compo- 

centered on the defined position of the fiduciary mark 42. nents process that finds groups of linked pixels, and applying 

The label decoding system 14 determines the position and a Hough transform th at finds line locations and orientations , 

orientation of the fiduciary mark 42 and uses that informa- ' trom the linked pixels. / 

tion to define the region of interest with respect to the Once the lines are found using the reduced resolution 

position of the fiduciary mark 42. The label decoding system 50 method, the original lines are restored to full resolution 

then creates and stores a high resolution text image within using the location information generated by the Hough 

the region of interest from the data captured by the high transform. Another connected components analysis is 

resolution camera 16. In this manner, only a relatively small applied to the full resolution lines in order to capture the text 

portion of the data captured by the high resolution camera 16 characters. Those skilled in the art will understand that 

is processed in order to decode the destination address data. 55 connected components analysis, and Hough transforms are 

This image is referred to as the region of interest (ROI) standard image processing techniques, 

image. Once the lines are identified, the method 308 proceeds to 

Although the system 10 locates, the destination address step 416 and performs a fine rotation on the characters 

block using the information provided by the fiduciary mark included in each line of the destination address. This fine 

detector 24, those skilled in the art will appreciate that 60 rotation completes the rotation process begun at step 412 

software techniques may be implemented to detect the and rotates the characters to horizontal (i.e., zero degrees), 

location and orientation of the destination address from the This ensures that the characters are properly oriented for the 

image provided by the high resolution OTB camera. Suit- application of the OCR algorithm, which attempts to decode 

able techniques would eliminate the need for the fiduciary each character in the destination address. This step is accom- 

mark detector, but would require additional computing 65 plished by applying forward rotational techniques. The 

resources in the label decoding system 14 Such software preferred rotational techniques are described by the follow- 

techniques may be used without departing from the spirit ing formulas: 
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where 4> is the orientation of the destination address after the 
coarse rotation performed at step 412. 

At step 418 the rotated characters are segmented or 
separated into separate characters. This is done because the 
OCR algorithm is applied to each character individually. At 
step 420 the OCR algorithm is applied to each of the 
characters in the destination address. Those skilled in the art 
will appreciate that the OCR algorithm uses a variety of 
techniques to recognize each characters and to determine 
what standard ASCII characters is represented by each 
character in the destination address. Those skilled in the art 
will also appreciate that the OCR algorithm may be used to 
decode other alphanumeric information on the package, 
such as the return address, shipper number, etc. A suitable 
OCR technique is described in U.S. Pat. No. 5,438,629, 
entitled "Method and Apparatus for Classification Using 
Non-spherical Neurons," which is incorporated herein by 
reference. 

At step 422 the OCR processed text is filtered to remove 
any characters that are not a part of the destination address. 

At step 424 the OCR processed destination address is 
validated or verified by attempting to match the decoded 
destination address with an address in the U.S. Postal 
Service's ZIP+4 database, which provides an exhaustive list 
of valid addresses in the United States. This step is necessary 
because the destination address and OCR algorithms do not 
include built in verification means such as checksums, etc. 

At step 426 the method 308 determines whether the 
decoded destination address matched a valid address in the 
ZIP+4 database or other database of valid addresses. If so, 
the method continues to step 428 where it returns to step 310 
of the method 300 (FIG. 3). Related methods for processing 
data in databases are described in commonly owned U.S. 
application Ser. No. 08/477,481, filed Jun. 7, 1995 and 
entitled "A Multi-Step Large Lexicon Reduction Method for 
OCR Application," which is incorporated herein by refer- 
ence. 

If the decoded address does not match a valid address in 
the ZIP+4 database, the method 308 proceeds to step 430 
and automatically attempts to correct common OCR errors 
in order to automatically provide a valid address. Typical 
OCR errors involve incorrectly decoding letters that look 
similar. Therefore, step 430 is optimized to correct OCR 
errors by substituting such letters in an attempt to match one 
of the valid addresses that appears in the address database. 

Those skilled in the art will understand that the validation 
process is tunable and involves three parameters. The accu- 
racy rate indicates the percentage of labels that are auto- 
matically read correctly. The error rate indicates the per- 
centage of labels that the system thinks it is has correctly, but 
are in fact incorrect. The rejection rate indicates the per- 
centage of labels that are not read correctly and which must 
be entered manually. The OCR validation process is tuned 
by first determining an acceptable error rate. Once this is 
determined, the system is tuned by adjusting the parameter 
that controls the relationship between the rejection rate and 
the error rate. 

At step 432 the method determines whether the substi- 
tuted characters have resulted in a valid address. If so, the 
method proceeds to step 428. 

If the method is unable to match correct the decoded 
address and match a valid address in the ZIP+4 database, the 
method proceeds to step 434 and transfers the image to a the 
image server 29, which is connected to one or more image 
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display workstations. The image display workstations dis- 
play an image of the destination address block and the 
closest possible addresses from the database. The image 
display workstation allows an operator to view the image of 

5 the destination address and manually enter the destination 
address into the workstation. This process (step 436) is 
described more completely in conjunction with FIG. 5. 

At step 438 the method 308 receives the manually entered 
destination address data from the image server. The infor- 

10 mation returned by the image server may take the form of 
manually entered address data or a selected one of the 
possible addresses from the database. After the address data 
is received from the image server, the method 308 proceeds 
to step 428 and returns to the method 300. 

is FIG. 5 is a flow diagram illustrating a method 500 carried 
out by the image server 29 and the image display worksta- 
tions 30o-c that form a part of the preferred system 10. As 
described above, the image display workstations are used to 
allow an operator to manually enter destination addresses 

20 that were not properly matched to valid addresses in the 
ZIP+4 database. This is accomplished by displaying an 
image of the destination address and the closest possible 
addresses from the database. The operator reads the address 
as it appears on the display and manually enters the address 

25 into the workstation or selects one of the displayed 
addresses. This manually entered address data is then 
returned to the label decoding system 14 where it replaces 
the improperly decoded OCR data. 
The method 500 begins at step 502 where the image 

30 server receives the image of the destination address from the 
label decoding system 14. The image server routes the image 
to a free image display workstation. At step 504 the image 
display workstation rotates the image to the nearest hori- 
zontal or vertical axis. At step 506 the rotated image is 

35 interpolated to form an image having a resolution of at least 
100 dots per inch (DPI) image, which is displayed at step 
508. In addition to the destination address image, the work- 
station also displays the closest possible matches from the 
ZIP+4 database. 

40 At step 510 the operator manually enters the destination 
address after having read the destination address presented 
on the display. The operator manually enters the correct 
destination address by selecting the correct address from the 
closest possible matches (if the correct address is displayed) 

45 or entering the address using a keyboard associated with the 
image display workstation. 

At step 512 the method determines whether the destina- 
tion address data entered by the operator was selected from 
the list of possible addresses selected from the database. If 

50 so, the method proceeds to step 514 and returns the correct 
destination address to the image server 29, which returns the 
data to the label decoding system 14. The method 500 then 
terminates at step 518. 

If at step 512 the method determines that the destination 

55 address data was typed in by the operator, the method goes 
to step 516 to validate the typed-in data. Those skilled in the 
art will appreciate that the error correction routine may be 
carried out at the image display workstation where the data 
was entered, at the image server after the data was returned 

60 from the image display workstation, or at a separate vali- 
dation computer connected to the image server via the 
network. 

Those skilled in the art will appreciate that the validation 
process of step 516 determines whether the keyed in address 
65 matches a valid address from the database. If not, the method 
also attempts to correct common key entry mistakes in order 
to see if the corrected key entered data matches one of the 
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addresses from the database. The validation/correction pro- 
cess is similar to the correction process described in con- 
junction with step 430 of FIG. 4, but is optimized for 
common key entry errors, which include substituting keys 
that are close together on the keyboard or letters that are 
transposed by the operator. The correction can be carried out 
by attempting to match a valid address from any address in 
the ZIP+4 database, or by trying to match one of the few 
close addresses transferred to the image display workstation 
from the label decoding system. 

After the manually entered destination address data is 
validated, the method proceeds to step 514 and returns the 
correct destination address to the image server 29, which 
returns the data to the label decoding system 14. The method 
500 then terminates at step 518. 

From the foregoing description, it will be appreciated that 
the present invention provides an efficient system and 
method for reading package information. The present inven- 
tion has been described in relation to particular embodi- 
ments which are intended in all respects to be illustrative 
rather than restrictive. Those skilled in the art will appreciate 
that many different combinations of hardware will be suit- 
able for practicing the present invention. Many commer- 
cially available substitutes, each having somewhat different 
cost and performance characteristics, exist for each of the 
components described above. 

Similarly, the method of the present invention may con- . 
veniently be implemented in program modules that are 
based upon the flow charts in FIGS. 3-5. No particular 
programming language has been indicated for carrying out 
the various procedures described above because it is con- 
sidered that the operations, steps and procedures described 
above and illustrated in the accompanying drawings are 
sufficiently disclosed to permit one of ordinary skill in the art 
to practice the instant invention. Moreover, there are many 
computers and operating systems which may be used in 
practicing the instant invention and therefore no detailed 
computer program could be provided which would be appli- 
cable to these many different systems. Each user of a 
particular computer will be aware of the language and tools 
which are most useful for that user's needs and purposes. 

Alternative embodiments will become apparent to those 
skilled in the art to which the present invention pertains 
without departing from its spirit and scope. Accordingly, the 
scope of the present invention is defined by the appended 
claims rather than the foregoing description. 

What is claimed is: 

1. A method for reading package information from a 
package, said package information including machine- 
readable first information indicia and alphanumeric second 
information indicia, comprising the steps of: 

capturing an image of said package, said image including 
said machine-readable first information indicia and said 
alphanumeric second information indicia; 

locating said machine-readable first information indicia in 
said image; 

automatically decoding said machine-readable first infor- 
mation indicia to provide package identification data; 

locating said alphanumeric second information indicia; 

automatically decoding said alphanumeric second infor- 
mation indicia to provide package destination data; 

combining at least a portion of said package identification 
data and at least a portion of said package destination 
data to form a unified package record; and 

affixing third information indicia to said package, said 
third information indicia being machine readable and 
comprising said unified package record. 

2. A method for reading package information as recited in 
claim 1, further comprising the step of storing said unified 
package record in a database. 
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3. A method for reading package information as recited in 
claim 1, further comprising the steps of: 

determining whether said package destination data is 
valid; 

5 displaying said image on a workstation; and 

receiving manually entered package destination data, and 
wherein said unified package record comprises said 
package identification data and said manually entered 
package destination data. 
10 4. A method for reading package information as recited in 
claim 3, wherein said manually entered package destination 
data comprises a destination address selected from a list of 
possible destination addresses displayed on said worksta- 
tion. 

j s 5. A method for reading package information as recited in 
claim 1, wherein locating said alphanumeric second infor- 
mation indicia comprises the steps of: 

identifying a mark indicative of the location of said 
alphanumeric second information indicia; and 
20 using said mark to locate said alphanumeric second 
information indicia. 

6. A method for reading package information as recited in 
claim 5, further comprising the step of rotating said alpha- 
numeric second information indicia. 

7. A system for reading package information from a 
2 package, said package information including machine- 
readable first information indicia and alphanumeric second 
information indicia, comprising: 

an imaging system including a camera for capturing an 

image of said package; 
a label decoding system for processing said image; and 
a printer for printing a label to be affixed to said package; 
said label decoding system being programmed to: 
locate said machine-readable first information indicia 
35 in said image; 

decode said machine-readable first information indicia 

to provide package identification data; 
locate said alphanumeric second information indicia; 
decode said alphanumeric second information indicia 
40 to provide package destination data; and 

combine said package identification data and said pack- 
age destination data to form a machine readable 
unified package record for printing on said label. 

8. A system for reading package information as recited in 
45 claim 7, wherein said label decoding system is further 

programmed to store said unified package record in a 
database. 

9. A system for reading package information as recited in 
claim 7, further comprising an image display workstation for 
displaying at least a portion of said image and for receiving 

50 manually entered data corresponding to said alphanumeric 
second information indicia, and wherein said label decoding 
system is further programmed to: 

determine whether said package destination data is valid; 
display said image on said workstation; and 
55 receive manually entered, package destination data, and 
wherein said unified package record comprises said 
package identification data and said manually entered 
package destination data. 

10. A system for reading package information as recited 
60 in claim 7, wherein locating said alphanumeric second 

information indicia comprises: 

identifying a mark indicative of the location of said 

alphanumeric second information indicia; and 
using said mark to locate said alphanumeric second 
65 information indicia. 

• * * * * 
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